Investigation  of  Cortical  Oscillation 
Models  Widiin  the  Visual  Cortex 


by  Adrienne  Raglin 
and  John  Johnson 


ARL-TR-1512 


October  1997 


SmC  QUALITY  IN8PBCTSD  2 


19971210  015 


Approved  for  public  release;  distribution  is  imlimited. 


The  hidings  in  this  report  are  not  to  be  construed  as  an  official 
Departmrat  of  the  Army  position  unless  so  designated  by  other 
authorized  documents. 

Citation  of  manufacturer’s  or  trade  names  does  not  constitute  an 
official  endorsemirat  or  approval  of  the  use  thereof. 

Destroy  this  report  when  it  is  no  longer  needed.  Do  not  return 
it  to  the  originator. 


Army  Research  Laboratory 

Georgia  Institute  of  Technology,  Atlanta,  GA  30332-0800 


ARL-TR-1512  October  1997 


Investigation  of  Cortical  Oscillation  Models 
Within  the  Visual  Cortex 


Adrienne  Raglin,  John  Johnson 

Corporate  Information  and  Computing  Center,  ARL 


Approved  for  public  release;  distribution  is  unlimited. 


Abstract 


Computer  vision  deals  with  algorithms  that  allow  machines  to  detect,  segment,  feature 
extract,  and  recognize  objects  in  an  image.  There  are  numerous  applications  in  medicine, 
manufacturing,  and  security  for  this  technology.  By  studying  the  visual  processes  of  biological 
systems,  enhancements  can  be  achieved  in  the  development  of  computer  vision  algorithms.  One 
biological  function  of  interest  involves  the  oscillatory  pulses  generated  in  the  primary  visual 
cortex  engaged  in  stimulus-specific  oscillatory  responses.  As  a  result  of  these  experiments,  it 
can  be  concluded  that  these  tightly  correlated,  stimulus-induced  oscillations  may  play  a  role  in 
the  recognition  of  images.  Therefore  these  cortical  oscillations  have  been  modeled  to  investigate 
their  ability  to  segment  objects  in  a  visual  field.  This  report  briefly  discusses  the  visual  system 
and  the  internally  stimulus-dependent  oscillations  that  may  lead  to  identification  of  images. 
Emphasis  will  be  on  the  models  that  attempt  to  reproduce  this  biological  phenomena,  their 
computational  and  behavioral  aspects,  as  well  as  simulation  performance.  Detail  will  be  given 
to  their  computational  and  behavioral  aspects  since  it  is  in  these  areas  that  possible 
improvements  can  be  achieved  through  more  detailed  modeling. 
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1.  Introduction 


Modeling  of  neurons  has  been  used  to  understand  or  enhance  the  performance  of  many  systems. 
Neuron  models  allow  us  to  map  out  the  processes  of  a  system,  to  visualize  the  flow  of  information 
and  to  investigate  the  effects  of  change  on  a  system,  and  are  even  used  as  a  diagnostic  tool  for  related 
systems. 

One  area  that  has  held  the  interest  of  scientists  and  laypersons  involves  our  own  visual  system. 
How  is  vision  achieved?  What  are  the  steps  in  the  visual  process?  Why  is  our  visual  system 
organized  in  the  manner  it  is?  Can  this  system  be  reproduced  in  a  nonbiological  medium?  In  the 
investigation  of  what  makes  our  visual  system  work,  scientists  have  observed  voltage  potentials 
generated  from  the  primary  cortex  located  in  our  brain.  These  potentials  oscillate  in  distinct  patterns 
when  different  stimuli  are  applied  to  our  eyes.  Can  these  oscillating  potentials  be  generated  outside 
of  biological  systems  to  allow  artificial  systems  to  “see”? 

In  this  report,  we  take  a  brief  look  at  the  anatomy  and  biological  testing  of  the  visual  system. 
Three  neuron  models  are  discussedt  the  general  model,  Eckhom’s  model,  and  Johnson’s  model. 

2.  General  Anatomy  of  the  Visual  System 

To  understand  how  the  visual  system  represents  images  in  the  visual  field  and  what  responses 
are  initiated  in  the  primary  visual  cortex,  we  must  understand  the  anatomy  of  the  visual  system  (see 
Figure  1).  The  components  of  the  visual  system  can  be  described  as  the  visual  field,  the  visual 
pathway,  and  the  primary  visual  cortex  (Kandell,  Schwartz,  and  Jessell  1991). 

The  visual  field  is  the  view  seen  by  two  eyes  without  movement  of  the  head.  The  visual  field 
projects  to  the  retina.  Cells  from  the  retina  form  the  optic  nerve.  Sections  of  the  optic  nerve  from 
each  eye  join  together  at  the  optic  chiasm,  and  from  there  the  nerves  continue  traversing  the  brain. 
Fibers  from  both  eyes  enter  the  optic  tract  and  project  to  the  lateral  geniculate  nucleus  (LGN),  which 
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is  a  section  of  the  cerebral  hemisphere.  LGN  cells  project  to  the  primary  visual  cortex.  The  primary 
visual  cortex  represents  the  visual  field  by  transforming  visual  information  into  linear  segments  and 
boundaries.  The  cells  found  in  the  primary  visual  cortex  project  to  other  areas  of  the  brain  where 
“vision”  is  processed.  Cells  in  the  primary  visual  cortex  respond  best  to  stimulus  that  is  linear  and 
has  a  specific  axis  of  orientation.  These  excited  neurons  or  cells  form  the  oscillations  seen  in  the 
visual  cortex. 


2 


Although  the  mechanism  for  the  oscillations  are  unknown,  it  is  suggested  that  the  interaction 
among  a  population  of  synaptically  coupled  neurons  are  sufficient  for  generating  oscillations. 
Experiments  also  show  that  when  activated  appropriately,  groups  of  adjacent  cortical  neurons  change 
in  cooperative  interactions.  These  interactions  lead  to  coherent  and  periodic  patterns  of  activity. 

3.  Modeling  of  Neuron 

3.1  General  Model.  A  variety  of  models  for  the  cortical  oscillations  found  in  the  visual  system 
have  been  proposed.  These  include  a  general  model  and  models  developed  by  Eckhom  and  Johnson 
(Eckhom  et  al.  1990;  Johnson  1994).  The  general  model  of  an  artificial  neural  net  (ANN)  (Vermuri 
1992)  is  an  artificial  neuron  (AN)  that  receives  inputs  from  a  number  of  other  ANs  or  from  an 
external  stimulus.  A  weighted  sum  of  these  inputs  constitutes  the  argument  to  a  function.  The 
resulting  value  of  the  function  is  the  output  of  the  AN.  The  output  mimics  the  firing  of  a  biological 
neuron.  This  ou^ut  gets  distributed  along  weighted  connections  to  other  ANs.  Therefore,  an  AN 
anatomically  models  a  biological  neuron. 

This  is  the  basis  for  modeling  of  the  neuron  and  neural  net.  However,  each  component  of  the 
model  requires  detail.  The  Johnson  and  Eckhom  models  recreate  the  neuronal  response  seen  in 
studies  of  the  primary  visual  cortex.  The  first  model,  Eckhom’s,  closely  recreates  the  responses  of 
the  biological  system.  The  second  model,  Johnson’s,  takes  Eckhom’s  model  and  attempts  to  modify 
it  to  use  the  patterns  of  the  neuronal  response  to  segment  images  in  a  visual  field. 

3.2  Eckhorn’s  Model.  The  stimulus-specific  response  is  a  series  of  oscillating  voltages  at 
approximately  53  Hz  and  60  Hz  (Johnson  1994).  The  findings  of  Grossberg  (1983)  and  Reitboeck 
(1983)  support  the  hypothesis  that  synchronization  may  be  a  mechanism  for  carrying  out  the  linking 
of  local  visual  features  into  impressions  of  objects. 

Stimulus-specific  synchronization  was  observed  in  different  cortical  areas  if  the  neurons  coded 
common  visual  features.  Signal  oscillations  are  generated  in  the  cortex  and  can  be  synchronized  by 
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stimulation  of  neurons  in  their  receptive  fields.  These  stimulus-specific  oscillations  are  the 
responses  modeled  by  Eckhom. 

The  stimulus-induced  synchronous  oscillatory  potential  found  to  take  place  in  the  visual  cortex 
were  observed  using  computer  simulations  of  neural  network  models.  Eckhom’s  model  (see 
Figure  2)  has  two  types  of  synapses:  the  feeding  synapse,  which  are  connected  directly  to  stimulus 
that  drive  the  neuron;  and  the  linking  synapses,  which  receive  auxiliary  signals  that  modulate  the 
input  from  the  feeding  synapses  (Eckhom  et  al.  1990).  The  linking  inputs  are  then  considered 
synchronizing  signals. 


dendrite  soma 


Figure  2.  Eckhom’s  Model. 


4 


The  model  neuron  has  dynamic  synapses  represented  by  leaky  integrators.  During  a  synaptic 
input  pulse,  the  integrator  is  charged  and  its  output  amphtude  rises  steeply.  This  is  followed  by  an 
exponential  decay  determined  by  the  time  constant.  The  spike  encoder  is  also  represented  by  a  leaky 
integrator  and  includes  a  differential  amplitude  discriminator  and  a  spike  former.  The  amplitude 
discriminator  triggers  the  spike  former  when  the  input  exceeds  a  variable  threshold.  The  output  of 
the  neuron  immediately  charges  the  leaky  integrator  and  raises  the  value  of  the  threshold.  This  new 
threshold  value  exceeds  the  input,  preventing  an  output  immediately  following  a  previous  output. 
The  elevated  threshold  produces  the  refractory  period  in  the  spike  generator,  so  that  another  output 
is  not  generated  until  the  threshold  value  decays  below  the  membrane  potential. 

Eckhom  believes  that  the  concept  of  a  modulatory  synapse  is  supported  by  modulation  seen  in 
real  neurons.  These  modulations  may  be  achieved  by  changing  dendritic  membrane  potentials 
through  voltage-dependent  channels  that  affect  synaptic  efficacy.  From  the  simulations,  feeding 
signals  with  modulation  causes  the  model  to  respond  initially  with  irregular  receptive  discharges  but, 
subsequently,  the  neurons  mutually  synchronize  their  activity  through  the  linking  connections.  This 
supports  the  phase  locking  of  stimulus-induced  synchronizations  in  the  simulation. 

3^.1  Computational  Aspects.  Eckhom’s  neural  network  consists  oftwo  layers  of  neurons.  The 
bottom  layer  (layer  1)  consists  of  several  neurons.  Each  layer  1  neuron  receives  inputs  from  the 
stimulus  as  feeding  inputs  and  input  from  the  four  closest  neighboring  neurons  as  linking  inputs. 
Layer  2,  the  top  layer,  also  consists  of  several  neurons.  Each  layer  2  neuron  receives  feeding  inputs 
from  the  four  closest  neurons  in  layer  1  and  four  linking  inputs  from  its  four  closest  neighboring 
neuron  in  layer  2.  Furthermore,  each  layer  2  neuron  sends  feeding  inputs  to  the  four  closest  layer  1 
neurons.  Eckhom  models  his  individual  neuron  as  a  dendrite  and  soma,  with  the  output  being  a 
pulse  and  the  input  being  fee  feeding  and  linking  inputs.  The  linking  inputs  and  feeding  inputs  are 
modeled  as  leaky  integrators.  Each  neuron  is  modeled  to  have  multiple  feeding  and  linking  inputs. 
The  linking  inputs  are  added  to  a  constant  value  of  1  then  multiplied  with  fee  feeding  input  to  form 
the  input  to  the  soma.  The  soma  is  modeled  as  a  spike  encoder.  The  spike  encoder  is  where  the 
ouqjut  from  the  dendrite  is  compared  with  a  preset  threshold  value.  If  the  value  is  greater  than  the 
threshold,  the  amplitude  discriminator  triggers  the  spike  former  to  output  a  pulse.  This  ouq)ut  is  fed 


5 


back  into  the  system  to  change  the  value  of  the  threshold  to  prevent  the  generating  of  additional 
pulses. 

3.2.2  Behavioral  Aspects.  Eckhom  believes  his  model  supports  synchronization  as  a  way  to 
achieve  perceptual  feature  linking.  To  bring  neuronal  mechanisms  of  feature  linking  into 
correspondence  with  perceptual  functions,  Eckhom  introduces  the  concept  of  a  local  neural  assembly 
linking  field.  This  linking  field  is  that  area  in  the  visual  space  where  appropriate  local  stimulus 
features  can  initiate  synchronizations  in  the  activities  of  that  assembly.  If  this  concept  can  be 
supported,  then  synchronized  patterns  of  neural  response  can  be  used  to  identify  visual  images.  It 
has  been  seen  that  sensory  systems  that  received  weU-timed  signals  might  be  dominated  by  stimulus- 
forced  synchronization.  Other  systems  may  employ  a  mechanism  of  synchronized  activity  for 
oscillations  and  coding  in  the  system. 

33  Johnson’s  Model.  Johnson’s  (1994)  model  is  a  modified  version  of  Eckhom’s  model.  It 
only  models  a  1-layer  network,  but  with  multiple  receptive  fields.  Johnson’s  model  includes  a 
dendritic  tree  section,  a  Hnking  section,  and  a  pulse  generator  section.  The  dendritic  tree  is  similar 
to  Eckhom’s  dendrite.  The  dendritic  tree  includes  the  feeding  inputs  from  other  neurons  as  well  as 
the  linking  inputs  fi-om  other  neurons.  The  difference  in  this  section  is  that  the  linking  inputs  are 
multiplied  by  a  variable  beta  before  the  constant  is  added.  Then  this  value  is  multiplied  with  the 
feeding  input  in  the  linking  section.  Moreover,  in  the  linking  section,  linking  inputs  from  other 
receptive  fields  are  multiplied  with  the  inputs  from  the  current  receptive  fields.  This  value  is  then 
summed  with  inputs  from  other  dendrites.  The  pulse  generator  section  includes  a  threshold 
discriminator  that  triggers  the  pulse  former  to  output  a  pulse  when  the  membrane  potential  exceeds 
the  threshold  value.  As  in  Eckhom’s  model,  the  output  is  fed  back  to  change  the  value  of  the 
threshold. 

33.1  Computational  Aspects.  It  is  helpful  to  look  at  an  individual  neuron  from  Johnson’s 
model  to  describe  what  functions  beta  and  tau-c  perform.  These  variables  are  experimental  and  have 
not  been  shown  to  model  biological  parameters.  Beta  is  called  the  weak  linking  variable.  It  sets  the 
strength  of  the  linking  inputs’  effect  on  the  feeding  inputs.  Tau-c  is  the  time  constant  that  defines 
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the  pulse  capture  zone.  This  zone  indicates  a  period  of  time  where  a  linking  input  can  influence  the 
frequency  of  the  output  pulses. 

In  this  model,  beta  (the  weak  linking  variable)  is  set  to  zero  when  there  is  no  feedback  from  the 
neighboring  neurons  (no  linking  input).  When  there  is  feedback,  beta  is  greater  than  zero.  To 
understand  this  concept,  the  ou^ut  of  a  single  neuron  with  beta  equal  to  zero  is  examined.  With  beta 
equal  to  zero,  the  membrane  potential  is  equal  to  the  feeding  input  only.  When  the  membrane 
potential  exceeds  the  threshold  value,  a  pulse  is  generated.  This  is  the  same  way  Eckhom’s  model 
operates. 

Now  consider  when  beta  is  greater  than  zero.  A  linking  input  can  add  to  the  membrane  potential, 
causing  the  membrane  potential  to  increase.  This  causes  a  pulse  to  occur  sooner  than  expected, 
implying  that  the  linking  input  (or  the  feedback  from  the  neighboring  neurons)  has  changed  when 
the  pulse  occurs.  The  linking  input  synchronizes  the  pulses  to  some  desired  pattern.  This  effect  of 
the  linking  input  is  also  found  in  Eckhom’s  model,  however,  the  linking  input  is  multiplied  by  a 
constant  of  1,  not  beta.  Thus,  in  Johnson’s  model,  the  linking  inputs  for  the  current  neuron  can  be 
arbitrarily  weighted  differently  from  linking  inputs  of  neurons  in  other  receptive  fields.  Since  these 
weights  are  not  from  biological  responses,  they  do  not  represent  synaptic  strengths  generated  by  a 
specific  stimulus.  Therefore,  this  model  does  not  use  these  weights  to  represent  data. 

3.32  Behavioral  Aspects.  Johnson  (1994)  proposes  that  the  periodic  pulses  that  are  generated 
by  a  particular  stimulus  can  be  used  to  identify  segments  of  an  image.  Each  small  section  of  an 
image  would  produce  its  individual  pulse  train  (see  Figure  3).  As  the  field  of  view  is  expanded,  it 
would  produce  its  individual  pulse  train  until  the  entire  scene  is  covered.  The  smaller  pulse  trams 
would  actually  become  subgroups  within  the  pulse  trains  produced  by  the  larger  section  of  the  scene. 
Therefore,  a  supergroup  pulse  train  would  represent  the  entire  scene  that  consists  of  subgroups  of 
pulse  trains  at  each  level  of  the  expanded  view. 
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Figure  3.  Behavior  of  Pulse  Train. 


4.  Conclusion 

This  report  has  presented  three  models  that  attempt  to  describe  the  oscillations  produced  in  the 
primary  visual  cortex.  Even  though  these  models  do  not  model  the  biological  system,  they  do  allow 
exploration  into  the  response  of  the  visual  system.  Further  modeling  can  be  done  to  incorporate 
more  of  the  functionality  of  the  real  biological  system.  These  enhancements  may  provide  a  more 
accurate  system  for  segmenting  objects. 
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